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, Abstract 

We give a detailed treatment of the "bit-model" of computability and complexity 
of real functions and subsets of M. n , and argue that this is a good way to formalize 
many problems of scientific computation. In Section ^ we also discuss the alternative 
Blum-Shub-Smale model. In the final section we discuss the issue of whether physical 
^ . systems could defeat the Church- Turing Thesis. 

(N 

S ■ 1 Introduction 



o 

The problems of scientific computing often arise from the study of continuous processes, and 
questions of computability and complexity over the reals are of central importance in laying 
the foundations for the subject. The first step is defining a suitable computational model 
for functions over the reals. 

Computability and complexity over discrete spaces have been very well studied since the 
1930s. Different approaches have been proved to yield equivalent definitions of computability 



and nearly equivalent definitions of complexity. From the tradition of formal logic we have 
the notions of recursiveness and Turing machine, and from computational complexity we 
have variations on Turing machines and abstract Random Access Machines (RAMs), which 
closely model actual computers. All of these converge to define the same well-accepted notion 
of computability. The Church- Turing thesis asserts that this formal notion of computability 
is broad enough, at least in the discrete setting, to include all functions that could reasonably 
construed to be computable. 

In the continuous setting, where the objects are numbers in JR., computability and com- 
plexity have received less attention and there is no one accepted computation model. Alan 
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Turing defined the notion of a single computable real number in his landmark 1936 paper 
|Tur36j : a real number is computable if its decimal expansion can be computed in the dis- 
crete sense (i.e. output by some Turing machine). But he did not go on to define the notion 
of computable real function. 

There are now two main approaches to modeling computation with real number inputs. 
The first approach, which we call the bit-model and which is the subject of this paper, reflects 
the fact that computers can only store finite approximations to real numbers. Roughly 
speaking, a real function / is computable in the bit model if there is an algorithm which, 
given a good rational approximation to x, finds a good rational approximation to f(x). 

The second approach is the algebraic approach, which abstracts away the messiness of 
finite approximations and assumes that real numbers can be represented exactly and each 
arithmetic operation can be performed exactly in one step. The complexity of a computa- 
tion is usually taken to be the number of arithmetic operations (for example additions and 
multiplications) performed. The algebraic approach applies naturally to arbitrary rings and 
fields, although for modeling scientific computation the underlying structure is usually R or 
C. Algebraic complexity theory goes back to the 1950s (see |BM75| IBCS97J for surveys). 

For scientific computing the most influential model in the algebraic setting is due to Blum, 
Shub and Smale (BSS) [BSS89J. The model and its theory are thoroughly developed in the 
book |BCSS98] (see also the article |Blum04j in the AMS Notices for an exposition). In the 
BSS model, the computer has registers which can hold arbitrary elements of the underlying 
ring R and performs exact arithmetic (+, — , •, and -r- in the case R is a field) and computer 
programs can branch on conditions based on exact comparisons between ring elements (=, 
and also <, in the case of an ordered field). Newton's method, for example, can be nicely 
presented in the BSS model as a program (which may not halt) for finding an approximate 
zero of a rational function, when R — R. A nice feature of the BSS model is its generality: 
the underlying ring R is arbitrary, and the resulting computability theory can be studied for 
each R. In particular, when R — Z 2 , the model is equivalent to the standard bit computer 
model in the discrete setting. 

One of the big successes of discrete computability theory is is the itncomputability results; 
especially the solution of Hilbert's 10th problem |Mat93j. The theorem states that there is 
no procedure (e.g. no Turing machine) which always correctly determines whether a given 
Diophantine equation has a solution. The result is convincing because of general acceptance 
of the Church- Turing thesis. 

A weakness of the BSS approach as a model of scientific computing is that uncomputabil- 
ity results do not correspond to computing practice in the case R — R. Since intermediate 
register values of a computation are rational functions of the inputs, it is not hard to see 
that simple transcendental functions such as e x are not explicitly computable by a BSS ma- 
chine. In the bit model these functions are computable because the underlying philosophy 
is that the inputs and outputs to the computer are rational approximations to the actual 
real numbers they represent. The definition of computability in the BSS model might be 
modified to allow the program to approximate the exact output values, so that functions like 
e x become computable. However formulating a good general definition in the BSS model 
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along these lines is not straightforward: see |Sma97j for an informal treatment and |Brv05j 
for a discussion and a possible formal model. 

For uncomputability results, BSS theory concentrates on set decidability rather than 
function computation. A set C C W 1 is decidable if some BSS computer program halts on 
each input i6K" and outputs either 1 or 0, depending on whether x G C. Theorem 1 in 
|BCSS98_ states that if C C W 1 is decided by a BSS program over R then C is a countable 
disjoint union of semi-algebraic sets. A number of sets are proved undecidable as corollaries, 
including the Mandelbrot set and all nondegenerate Julia sets (Fig. |2J). However again it 
is hard to interpret these undecidability results in terms of practical computing, because 
simple subsets of M? which can be easily "drawn" , such as the Koch snowflake and the graph 
of y = e x (Fig. |TJ) are undecidable in this sense [Brt03 





Figure 1: The Koch snowflake and the graph of the equation y = e x 

In the bit model there is a nice definition of decidability (bit-computability) for bounded 
subsets of W 1 ( Section |3.3|) . For the case of 1R 2 , the idea is that the set is bit-computable if 
some computer program can draw it on a computer screen. The program should be able to 
zoom in at any point in the set and draw a neighborhood of that point with arbitrarily fine 
detail. Such programs can be easily written for simple sets such as the Koch snowflake and 
the graph of the equation y = e x , and more sophisticated programs can be written for many 
Julia sets (Section 0J) . A Google search on the World Wide Web turns up programs that 
apparently do the same for the Mandelbrot set. However no one knows how accurate these 
programs are. The bit computability of the Mandelbrot set is an open question, although 
it holds subject to a major conjecture (Section HJ). Because of the Church- Turing thesis, a 
proof of bit-uncomputability of the Mandelbrot set would carry some force: the programs 
purporting to draw it must draw it wrong, at least at some level of detail. 

In the rest of the paper we concentrate on the bit model, because we believe that this 
is the most accurate abstraction of how computers are used to solve problems over the 
reals. The bit model is not widely appreciated in the mathematical community, perhaps 
because its principal references are not written to appeal to numerical analysts. In contrast 
the BSS model has received widespread attention, partly because its presentation BSS89J, 



3 



and especially the excellent reference [BCSS98J, not only present the model but provide a 
rigorous treatment of matters of interest in scientific computation. The authors deserve 
credit not just for presenting an elegant model, but also for arousing interest in foundational 
issues for numerical analysis, and inspiring considerable research. Undoubtedly the concept 
of abstracting away round-off error in computations over the reals poses important natural 
questions. One example is linear programming: Although polynomial time algorithms are 
known for solving linear programming problems in the bit model, no such algorithm is known 
for the BSS model. The commonly-used simplex algorithm can be neatly formalized in the 
BSS model, but it requires exponential time in the worst case. It would be very nice to find 
a polynomial time BSS algorithm for linear programming. 

Here is an outline of the remaining sections. Section |21 gives examples of easy and hard 
computational problems over the reals. Section|H]motivates and defines the bit model both for 
computing real functions and subsets of W n . Computational complexity issues are discussed. 
Section |3] treats the computability and complexity of the Mandelbrot and Julia sets. Simple 
programs are available which seem to compute the Mandelbrot set, but they may draw pieces 
which should not be there. The bit-computability of the Mandelbrot set is open, but it is 
implied by a major conjecture. Many Julia sets are not only computable, but are efficiently 
computable (in polynomial time). On the other hand some Julia sets are uncomputable 
in a fundamental sense. Finally Section |3] discusses a fundamental question related to the 
Church- Turing thesis: are there physical systems that can compute functions which are 
uncomputable in the standard computer model? 

Some of the material presented here is given in more detail in [Brv05j. See t Ko95 and 
[WciOO for general references on bit-computability models. 

Acknowledgments The authors are grateful to the following people for helpful com- 
ments on a preliminary version of this paper: Eric Allender, Lenore Blum, Peter Hertling, 
Ken Jackson, Charles Rackoff, Klaus Weihrauch, and Michael Yampolsky. 

2 Examples of "easy" and "hard" problems 
2.1 "Easy" problems 

We start with examples of problems over the reals that should be "easy" according to any 
reasonable model. The everyday operations, such as addition, subtraction and multiplication 
should be considered easy. More generally, any of the operations that can be found on a 
common calculator can be regarded as "easy", at least in some reasonable region. Such 
functions include, among others, shire, e x , y/x, and logx. 

Functions with singularities, such as x -r- y and tan x are easily computable on any closed 
region which excludes the singularities. The computational problem usually gets harder 
as we approach the singularity point. For example, computing tanx becomes increasingly 
difficult as x tends to |, because the expression becomes increasingly sensitive in x. 

Some basic numerical problems that are known to have efficient solutions should also be 
relatively "easy" in the model. This includes inverting a well conditioned matrix A. A matrix 
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is well conditioned in this setting if A 1 does not vary too sharply under small perturbations 
of A. 

Simple problems that arise naturally in the discrete setting should usually remain simple 
in the continuous setting. This includes problems such as sorting a list of real numbers and 
finding shortest paths in a graph with real edge lengths. 

When one considers subsets of M 2 , a set should be considered "easy", if we can draw it 
quickly with an arbitrarily high precision. Examples include simple "paintbrush" shapes, 
such as the ball B((0, 0), 2) in IR 2 , as well as simple fractal sets, such as the Koch snowflake 
(Fig. EJ. 

To summarize, the model should classify a problem as "easy", if there is an efficient 
algorithm to solve it in some practical sense. This algorithm may be inspired by a discrete 
algorithm, a numerical-analytic technique, or both. 

2.2 "Hard" problems 

Naturally, the "hard" problems are the ones for which no efficient algorithm exists. For 
example, it is hard to compute an inverse A~ l of a poorly conditioned matrix A. Note that 
even simple numerical problems, such as division (x + 1) -v- (y — 1), become increasingly 
difficult in the poorly conditioned case. It becomes increasingly hard to evaluate the latter 
expression as y approaches 1. 

Many problems that appear to be computationally hard arise while trying to model 
processes in nature. A famous example is the iV-body problem, which simulates the motion 
of planets. An even harder example is solving the Navier-Stokes equations used in simulations 
for fluid mechanics. We will return to questions of hardness in physical systems in Section 

El 

Another class of problems that should be hard is natural extensions of very difficult 
discrete problems. Consider, for example, the subset sum problem. In this problem we are 
given a list S = {a 1; a 2 , . . . , a n } of numbers and a number A. Our goal is to decide whether 
there is a subset {a^, a i2 , . . . ,a ik } that adds up to A, i.e. + a i2 + . . . + a ik = A. This 
problem is widely believed not to have an efficient solution in the discrete case. In fact it 
is iVP-complete in this case f |GJ79j ). and having a polynomial time algorithm for it would 
imply that P = NP, which is believed to be unlikely. There is no reason to think that it 
should be any easier in the continuous setting than in the discrete case. 

The hardness of numerical problems may significantly vary with the domain of applica- 
tion. Consider for example the problem of computing the integral I(x) = Jq f(t)dt. While 
it is very easy to compute I(x) from f(x) in the case / is a polynomial, it can be extremely 
difficult in the case / is a more general efficiently computable function. 

2.3 Quasi- fractal examples: the Mandelbrot and Julia sets 

The Mandelbrot and the Julia sets are common examples of computer-drawn sets. Beautiful 
high-resolution images have become available to us with the rapid development of computers. 
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Amazingly, these extremely complex images arise from very simple discrete iterated processes 
on the complex plane C. 

For a point c G C, define f c {z) = z 2 + c. c is said to be in the Mandelbrot set M if 
the iterated sequence c, / c (c), / c (/c(c)), . . . does not diverge to oo. While (we believe) very 
precise images of M can be generated on a computer, proving that these images approximate 
M would probably involve solving some difficult open questions about it. 

The family of Julia sets is parameterized by functions. In the simple case of quadratic 
functions f c (z) = z 2 + c, the filled Julia set K c is the set of points x, such that the sequence 
x, f c (x), f c (fc( x )), ■ ■ ■ does not diverge to oo. The Julia set J c is defined as the boundary of 
K c . While many Julia sets, such as the ones in Fig. are quite easy to draw, there are 
explicit sets of which we simply cannot produce useful pictures. 




Figure 2: a: the Mandelbrot set; b-d: Julia sets with parameter values of i, —1.57 and 
0.5 + 0.56i, respectively. 

As we see, it is not a priori clear whether these sets should be "easy" or "hard" . This 
gives rise to a series of questions: 

• Is the Mandelbrot set computable? 

• Which Julia sets are computable? 

• Can the Mandelbrot sets and its zoom-ins be drawn quickly on a computer? 



6 



• Which Julia sets and their zoom-ins can be drawn quickly on a computer? 

These questions are meaningless unless we agree upon a model of computation for this 
setting. In the following sections we develop such a model, based on "drawability" on a 
computer. 



3 The bit model 

3.1 Bit computability for functions 

The motivation behind the bit model of computation is idealizing the process of scientific 
computing. Consider for example the simple task of computing the function iwe 1 for an x 
in the interval [—1,1]. The most natural solution appears to be by taking the Taylor series 
expansion around 0: 

OO 

k=0 

Obviously in a practical computation we will only be able to add up a finite number of terms 
from (|TJ. How many terms should we consider? By taking more terms, we can improve the 
precision of the computation. On the other hand, we pay the price of increasing the running 
time as we consider more terms. The answer is that we should take just enough terms to 
satisfy our precision needs. 

Depending on the application, we might need the value of e x within different degrees of 
precision. Suppose we are trying to compute it with a precision of 2~ n . That is, on an input 
x we need to output a y such that \e x — y\ < 2" n . It suffices to take n + 1 terms from (JTJ) to 
achieve this level of approximation. Indeed, assuming n > 4, for any x G [—1, 1], 



Ex 

k=0 



EX 

k=n+l 



< 



OO I I oo . oo 

y \lA< y I< y 



k=n+l 



k=n+l 



k=n+l 



1 

2k+i 



>+i) 



In fact, a smaller number of terms suffices to achieve the desired precision. We take a portion 
of the series that yields error 2 - ( n+1 ) < 2~ n , to allow room for computation (round-off) errors 
in the evaluation of the finite sum. All we have to do now is to evaluate the polynomial 
Pn{%) — Xlfc=o ff w ithin an error of 2~( n+1 \ To do this, we need to know x with a certain 
precision 2~ m . It is convenient to use dyadic rationals to express approximations to x and 
e x , where the dyadic rationals form the set 



D = {§ | m e Z,n e N}. 

We can then take a dyadic q = ^ e D such that \x — q\ < 2~ m , and evaluate p n (q) within 
an error of 2~( n+2 ) using finite precision dyadic arithmetic. This gives us an approximation 
y E Dofp n (g) such that \y—p n (q)\ < 2~ ( - n+2 \ In our example, an assumption \x—q\ < 2~( n+4 ) 
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guarantees that \p n (x) — p n (q)\ < 2 ( n+2 \ and we can take m = n + 4. To summarize, we 
have 

\e x -y\ < \e x - p n (x)\ + \p n (x) - p n (q)\ + \p n (q) - y\ < 2'^ + 2'^ + 2'^ = 2~ n , 

and y is the answer we want. The running time of the computation is dominated by the 
time it takes to compute our approximation to p n (q). Note that this entire computation is 
done over the dyadic rationals, and can be implemented on a computer. 

To find the answer we only needed to know x within an error of 2 _(n+4 ). This is especially 
important when one tries to compose several computations. For example, to compute e e *~ l 
within en error 2~ n on the appropriate interval, one would need to know x within an error 

of2 -(«+8). 

While evaluating the function e x is hardly a challenge for scientific computing, the process 
described above illustrates the main ideas in the bit model of computation. Below are the 
main points that we have seen in this example, and that characterize the bit model for 
computing functions. Suppose we are trying to compute / : [a, b] — > M. n . Denote the 
program computing / by Mf. 

• The goal of Mf(x,n) is to compute f(x) within the error of 2" n ; 

• Mf computes a precision parameter m, it needs to know x within an error of 2~ m to 
proceed with the computation; 

• Mf receives a dyadic q = ^ such that \x — q\ < 2 _m ; 

• Mf computes a dyadic y such that \f(x) — y\ < 2~ n ; 

• the running time of Mf(x,n) is the time it takes to compute m plus the time it takes 
to compute y from q (both of which have finite representations by bits). 

Note that the entire computation of Mf is performed only with finitely presented dyadic 
numbers. There is a nice way to present the computation using oracle terminology. An 
oracle for a real number x is a function : N — > D such that for all n, \<f>(n) — x\ < 2~ n . Note 
that the q in the description above can be taken to be q = <f>(m). Instead of querying the 
value of x once, we can allow Mf an unlimited access to the oracle 0. The only limitation 
would be that the time it takes Mf to read the output of 4>(m) is at least m. We emphasize 
the fact that x is presented to Mf as an oracle by writing Mf(n) instead of Mf(x, n). Just as 
the quality of the answer of Mf(x,n) should not depend on the specific 2~ m -approximation 
q for x, Mf(n) should output a 2~™-approximation of f(x) for any valid oracle for x. 

The running time T(n) of Mj(n) is the worst case time a computation on this machine 
can take with a valid input and precision n. If T(n) is bounded by some polynomial p(n), 
we say that Mj? works in polynomial time. 

The output of Mf(n) can be viewed itself as an oracle for f(x). This allows us to compose 

functions. For example, given a machine Mj(n) for computing f(x), and a machine M^{n) 

for computing g(x), we can compute / o g(x) by M, 9 (n). 
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This is the bit-computability notion for functions. It was first proposed by Grzegorczyk 
|Grz55| and Lacombe |Lac55j . It has been since developed and generalized. More recent 
references on the subject include [Ko91 , PR89J, and jWeiOOj . Let us see some examples to 
illustrate this notion. 

3.2 Examples of bit-computability 

We start with a family of the simplest possible functions, the constant functions. For a 
function f(x) = a, a G M a constant, we can completely ignore the input x. The complexity 
of computing f(x) with precision 2~ n is the complexity of computing the number a within 
this error. In the original work by Turing |Tur36j . the motivation for introducing Turing 
machines was defining which numbers can be computed, and which cannot. 

For example, the numbers ~ = (0.01010101 .. .) 2 and \[2 appear to be "easy", with 
the latter being "harder" than the former. There are also easy algorithms for computing 
transcendental numbers such as ir and e. But there are only countably many programs, hence 
all but countably many reals cannot be approximated by any of them. To give a specific 
example of a very hard number, consider some standard encoding of all the Diophantine 
equations, : {equations} — > N. Let D = {(p(EQ) : EQ is a solvable equation} C N, and 

d = Y, 4 ~ n - 

Then computing d with an arbitrarily high precision would allow us to decide whether <p(EQ) 
is in D for any specific Diophantine equation EQ. This would contradict the solution to 
Hilbert's 10th Problem, which states that no such decision procedure can exist (see [Mat93j, 
for example). 

The following example will illustrate the bit-computability of a more general function. 
Example: Consider the function f(x) = \/l — x 3 on the interval [0, 1]. It is a composition 
of two functions: g : x i— > 1 — x 3 and h : x i— > ^/x, both on the [0, 1] interval, g is 
easier to compute in this case. To obtain g(x) with precision 2~ n it suffices to query for an 
approximation q E [0,1] of x with precision 2 _<n+2 \ and to return a = 1 — q 3 , computed 
within an error of 2~( n+2 \ Note that the second operation is done entirely with bits. We 
obtain an answer a such that 

\a-g(x)\ < \a-g(q)\ + \q 3 -x 3 \ < 2~ (n+2) + \q-x\ ■ \q 2 + qx + x 2 \ < 2" (n+2) + 2" (n+2) -3 = 2~ n . 

The function h(x) is slightly trickier to compute. One possible approach is to use Newton's 
method to solve (approximately) the equation A 3 — x = to obtain A = ^fx. The fact that 
g(x) and h(x) are computable is not surprising. In fact, both these functions can be found 
on a common scientific calculator. 

In general, all "calculator functions" are computable on a properly chosen domain. For 
example, x t— > 1/x is computable on any domain which excludes 0. We can bound the time 
required to compute the inverse, if the domain is properly bounded away from 0. The only 
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true limiting factor here is that computable functions as described above must be continuous 
on the domain of their application. This is because the value of f(x) we compute must be 
a good approximation for all points near x. 

Theorem 1 Suppose that a function f : S — > M fc is computable. Then f must be continuous. 

This puts a limitation on the applicability on the computability notion above. While it is 
"good" at classifying continuous functions, it classifies all discontinuous functions, however 
simple, as being uncomputable. We will return to this point later in Section ET31 

3.3 Bit-computability for sets 

Just as the bit-computability of functions formalizes finite-precision numerical computation, 
we would like the bit-computability of sets to formalize the process of generating images of 
subsets in M fc , such as the Mandelbrot and Julia sets discussed in Section I2~31 

An image is a collection of pixels, which can be taken to be small balls or cubes of size e. 
e can be seen as the resolution of the image. The smaller it is, the finer the image is. The 
hardness of producing the image generally increases as e gets smaller. A collection of pixels 
P is a good image of a bounded set S if the following conditions are fulfilled: 

• P covers S. This ensures that we "see" the entire set P, and 

• P is contained in the e-neighborhood of S. This ensures that we don't get "irrelevant" 
components which are far from S. 

It is convenient to take e — 4 • 2~ n - our computation precision in this case. 

Suppose now that we are trying to construct P as a union of pixels of radius 2~ n centered 
at grid points (2 -(n+fc ) • Z) fc . The basic decision we have to make is whether to draw each 
particular pixel or not, so that the union P would satisfy the conditions above. To ensure 
that P covers S, we include all the pixels that intersect with S. To satisfy the second 
condition, we exclude all the pixels that are 2 _ra -far from S. If none of these conditions 
holds, we are in the "gray" area, and either including or excluding a pixel is fine. In other 
words, we should compute a function from the family 



for every point d E (2~( n+fc ) ■ Z) fc . Here / is computed in the classical discrete sense. 
Example: A "simple" set, such as a point, line segment, circle, ellipse, etc. is computable if 
and only if all of its parameters are computable numbers. For example, a circle is computable 
if and only if the coordinates of its center and its radius are computable. 

The way we have arrived at the definition of bit-computability might suggest that it is 
specifically tailored to computer-graphics needs and is not mathematically robust. This is 




or 1 otherwise 




(2) 
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not the will be seen in the following theorem. Recall that the Hausdorff distance 

between bounded subsets of M. k is defined as 



d H (S,T) = mf{d : ScB(T,d), and T C B(S, d)}. 
We have the following. 

Theorem 2 Let Scl* be a bounded set. Then the following are equivalent. 

1. A function from the family HJ) is computable. 

2. There is a program that for any e > gives an e- approximation of S in the Hausdorff 
metric. 

3. The distance function ds{x) = infjjx — y\ : y GS"} is bit- computable. 
1. and 3. remain equivalent even if S is not bounded. 

Example: The finite approximations of the Koch snowflake are polygons that are ob- 
viously computable. The convergence — > K is uniform in the Hausdorff metric. So K 
can be approximated in the Hausdorff metric with any desired precision. Thus the Koch 
snowflake is bit-computable. 

The last characterization of set bit-computability in Theorem |21 connects the computabil- 
ity of sets and functions. There is another natural connection between the computability 
notions for these objects - through plots of a function's graph. 

Theorem 3 Let D C M. k be a closed and bounded computable domain, and let f : D —>■ R 
be a continuous function. Then f is computable as a function if and only if the graph 
Tf = {(x, f(x)) : x G D} is computable as a set. 

Example: Consider the set S = {(x,y) : x,y G [0, 1], x 3 + y 3 = 1}. It is the graph of the 
function f(x) = ^1 — x 3 on [0, 1], which has been seen to be computable in Section By 
Theorem El S is a bit-computable set. This is despite the fact (pointed out in |Blum04j ) 
that the only rational points in S are (0, 1) and (1, 0). 

The bit computability notion dates back to Lacombe |Lac58j . We refer the reader to 
B W99| IWeiOOt IBrv05j for a more detailed discussion. 



3.4 Computational complexity in the bit model 

Since the basic object in the discussion above is a Turing Machine, the computational com- 
plexity for the bit model follows naturally from the standard notions of computational com- 
plexity in the discrete setting. Basically, the time cost of a computation is the number of bit 
operations required. 

For example, the time complexity T n (n) for computing the number 7r is the number of 
bit operations required to compute the first n binary digits of a 2 _ra -approximation of 7r. 
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The time complexity T e x(n) of computing the exponential function x \— > e x on [— 1, 1], is the 
number of bit operations it takes to compute a 2 _n -approximation of e x given an x G [—1, 1]. 
This running time is assessed at the worst possible admissible x. We have seen that both 
T^-(n) and T e x{n) are bounded by a polynomial in n. 

This computational complexity notion can be used to assess the hardness of the different 
numerical-analytic problems arising in scientific computing. For example, the dependence of 
matrix inversion hardness on the condition number of the matrix fits nicely into this setting. 

Schonhage Sch82, Sch8?j has shown how the fundamental theorem of algebra can be 
implemented by a polynomial time algorithm in the bit model. More precisely, he has shown 
how to solve the following problem in time 0((n 3 + n 2 s) log 3 (ns)): Given any polynomial 
P{z) = a n z n + . . . + do with aj G C and \P\ = ^2 \a v \ < 1, and given any s G N, find 
approximate linear factors Lj(z) = UjZ + Vj (1 < j < n) such that \P — L1L2 ■ ■ ■ L n \ < 2~ s 



The complexity of computing a set is the time T(n) it takes to decide one pixel. More 
formally, it is the time required to compute a function from the family ((21). Thus a set is 
polynomial-time computable if it takes time polynomial in n to decide one pixel of resolution 



To see why this is the "right" definition, suppose we are trying to draw a set S on a 
computer screen which has a 1000 x 1000 pixel resolution. A 2~ n -zoomed in picture of S has 
0(2 2n ) pixels of size 2~ n , and it would take time 0{T{n) ■ 2 2n ) to compute. This quantity 
is exponential in n, even if T(n) is bounded by a polynomial. But we are drawing S on a 
finite-resolution display, and we will only need to draw 1000 • 1000 = 10 6 pixels. Hence the 
running time would be O(10 6 -T(n)) = 0(T(n)). This running is polynomial in n if and only 
if T(n) is polynomial. Hence T(n) reflects the 'true' cost of zooming in, when drawing S. 

3.5 Beyond the continuous case 

As we have seen earlier, the bit model notion of computability is very intuitive for sets and 
for continuous functions. However, by Theorem ^ h completely excludes even the simplest 
discontinuous functions. Consider for example the step function 



The function is bit computable on any domain which excludes 0. One could make the 
argument that a physical device really cannot compute Sq. There is no bound on the precision 
of x needed to compute sq(x) near 0. Hence no finite approximation of x suffices to compute 
So even within an error of 1/3. 

On the other hand, one might want to include this function, and other simple functions in 
the computable class, because the primary goal of this classification is to distinguish between 
"easy" and "hard" problems, and computing sq does not look hard. One possibility is to say 
that a function is computable if we can plot its graph. By Theorem |SJ this definition extends 
the standard bit-computability definition in the continuous case. It is obvious that is makes 



holds. 



2 -n_ 




(3) 
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So computable, since the graph of Sq is just a union of two rays. The notion of computational 
complexity can be extended into this more general setting as well. More details can be found 
in [BrvOft] . 



4 How hard are the Mandelbrot and Julia sets? 

First let us consider questions of computability of the Mandelbrot set M (which was defined 
in Section ESI)- Despite the relatively simple process defining M, the set itself is extremely 
complex, and exhibits different kinds of behaviors as we zoom into it. In Fig. |3]we see some 
of the variety of images arising in M. 




Figure 3: A variety of different images arising in zoom-ins of the Mandelbrot set 

The most common algorithm used to compute M is presented on Fig. The idea is to 
fix some number T, which is the number of steps for which we are willing to iterate. Then 
for every gridpoint c iterate f c (z) = z 2 + c on c for at most T steps. If the orbit escapes 
5(0, 2), we know that c ^ M. Otherwise, we say that c G M. This is equivalent to taking 
the inverse image of -6(0, 2) under the polynomial map f T (c) = f c o f c o . . . o / c (c). In Fig. 

t times 

0] (right) a few of these inverse images and their convergence to M are shown. 

One problem with this algorithm is that its analysis should take into account roundoff 
error involved in the computation z <— z 2 + c. But there are other problems as well. For 
example, we take an arbitrary grid point c to be the representative of an entire pixel. If 
c is not in M, but some of its pixel is, we will miss this entire pixel. This problem arises 
especially when we are trying to draw "thin" components of M, such as the one in the center 
of Fig. BJ 

Perhaps a deeper objection to this algorithm is the fact that we do not have any estimate 
on the number of steps T(n) we need to take to make the picture 2 _?l -accurate. That is, a 
T(n) such that for any c which is 2~"-far from M, the orbit of c escapes in at most T{n) 
steps. In fact, no such estimates are known in general, and the questions of their existence 
is equivalent to the bit computability of M (cf. |Hert0 5 j). 

Some of the most fundamental properties of M remain open. For example, it is conjec- 
tured that it is locally connected, but with no proof so far. 
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Conjecture 4 The Mandelbrot set is locally connected. 

When one looks at a picture of M, one sees a somewhat regular structure. There is a 
cardioidal component in the middle, a smaller round component to the left of it and two even 
smaller components on the sides of the main cardioid. In fact, many of these components 
can be described combinatorially based on the limit behavior of the orbit of c. E.g., in the 
main cardioid, the orbit of c converges to an attracting point. These components are called 
hyperbolic components because they index the hyperbolic Julia sets that will be discussed 
below. Douady and Hubbard [DH§2] have shown that Conjecture 0] implies that the interior 
of M consists entirely of hyperbolic components. 

Conjecture 5 The interior of M is equal to the union of its hyperbolic components. 

The latter conjecture is known as the Density of Hyperbolicity Conjecture. Hertling 
|Hert05j has shown that the Density of Hyperbolicity Conjecture implies the computability 
of M. There is a possibility that M is computable even without this conjecture holding, 
but it is hard to imagine such a construction without a much deeper understanding of the 
structure of M. Moreover, even if M is computable, questions surrounding its computational 
complexity remain wide open. As we will see, the situation is much clearer for most Julia 
sets. 

A Julia set J r is defined for every rational function r from the Riemann sphere into itself. 
Here we restrict our attention to Julia sets corresponding to quadratic polynomials of the 
form fc(z) = z 2 + c. Recall that the filled Julia set K c is the set of points x such that 
the sequence x, f c (x), f c (f c (x)), . . . does not diverge to oo. The Julia set J c = dK c is the 
boundary of the filled Julia set. 

For every parameter value c there is a different set J c , so in total there are uncountably 
many Julia sets, and we cannot hope to have a machine computing J c for each value of c. 
What we can hope for is a machine computing J c when given an access to the parameter 
c with an arbitrarily high precision. The existence of such a machine and the amount of 
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time the computation takes depends on the properties of the particular Julia set. More 
information on the properties of Julia sets can be found in [MilOO . 

Computationally, the "easiest" case is that of the hyperbolic Julia sets. These are the sets 
for which the orbit of the point either diverges to oo or converges to an attracting orbit. 
Equivalently, these are the sets for which there is a smooth metric /i on a neighborhood 
iV of J c such that the map f c is strictly expanding on N in \i. Hence, points escape the 
neighborhood of J c exponentially fast. That is, if d(J c , x) > 2~ n , then the orbit of x will 
escape some fixed neighborhood of J c in 0(n) steps. This gives us the divergence speed 
estimate we lacked in the computation of the Mandelbrot set M, and shows that in this case 
J c is computable in polynomial time (see HWO.'U lBrv04| lRet04j for more details). The set M 
can be viewed as the set of parameters c for which J c is connected. The hyperbolic Julia sets 
correspond to the values of c which are either outside M or in the hyperbolic components 
inside M. If Conjecture holds, all the points in the interior of M correspond to hyperbolic 
Julia sets as well. None of the points in dM correspond to hyperbolic Julia sets. 

We have just seen that the most "common" Julia sets are computable relatively efficiently 
These are the Julia sets that are usually drawn, such as the ones on Fig. |21 This raises the 
question of whether all Julia sets are computable so efficiently. The answer to this question 
is negative. In fact, it has been shown in [BY04J that there are some values of c for which 
J c cannot be computed (even with oracle access to c). Moreover, in |BBY05j it has been 
shown that a computable J c with an explicitly computable c can have an arbitrarily high 
computational complexity. The constructions are based on Julia sets with Siegel disks. A 
parameter c which "fools" all the machines attempting to compute J c is constructed, via 
a diagonalization similar to the one used in other noncomputability results. Thus, while 
"most" Julia sets are relatively easy to draw, there are some whose pictures we might never 
see. 

5 Computational hardness of physical systems and the 
Church- Turing Thesis 

In the previous sections we have developed tools which allow us to discuss the hardness of 
computational problem in the continuous setting. As in the discrete case, true hardness of 
problems depends on the belief that all physical computational devices have roughly the 
same computational power. In this section we present a connection between tractability of 
physical systems in the bit model, and the possibility of having computing devices more 
powerful than the standard computer. This provides further motivation for exploring the 
computability and computational complexity of physical problems in the bit model. The 
discussion is based in part on [Yao02j. 

The Church- Turing thesis (CT), in its common interpretation, is the belief that the 
Turing machine, which is computationally equivalent to the common computer, is the most 
general model of computation. That is, if a function can be computed using any physical 
device, then it can be computed by a Turing machine. 
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Negative results on in computability theory depend on the Church- Turing thesis to give 
them strong practical meaning. For example, by Hilbert's 10th Problem [Mat 93] , Diophan- 
tine equations cannot be generally solved by a Turing Machine. This implies that this 
problem cannot be solved on a standard computer, which is computationally equivalent to 
the Turing Machine. We need the CT to assert that the problem of solving these equations 
cannot be solved on any physical device, and thus is truly hard. 

When we discuss the computational complexity of problems, we are not only interested in 
whether a function can be computed or not, but also in the time it would take to compute 
a function. The Extended Church- Turing thesis (ECT) states that any physical system is 
roughly as efficient as a Turing machine. That is, if it computed a function / in time T(n), 
then / can be computed by a Turing Machine in time T(n) c for some constant c. 

In recent years, the ECT has been questioned, in particular by advancements in the theory 
of quantum computation. In principle, if a quantum computer could be implemented, it would 
allow us to factor an integer N in time polynomial in logiV [Shor97 . This would probably 
violate the ECT, since factoring is believed to require superpolynomial time on a classical 
computer. On the other hand there is no apparent way in which quantum computation 
would violate the CT. 




Figure 5: Computing / using a "hard" physical device A (left); a fair coin cannot be con- 
sidered a "hard" device 

Let / be some uncomputable function. Suppose that we had a physical system A and 
two computable translators <fi and if), such that <ft translates an input x to / into a state 
4>(x) of A. The evolution of A on cr = <p(x) should yield a state o~t such that t/>(ctt) = f(x) 
(Fig. EJ). This means that at least in principle we should be able to build a physical device 
which would allow us to compute an uncomputable function. To compute / on an input x, 
we translate x into a state <7o = (f>(x) of A. We then allow A to evolve from state ctq to ot - 
this is the part of the computation that cannot be simulated by a computer. We translate 
o"t to obtain the output f(x) = i/)((Jt)- 
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To make this scheme practical, we should require A to be robust around ctq = 4>(x), at 
least in some probabilistic sense. That is, for a small random perturbation <jq + e of <tq, 
■0(ct) should be equal to f(x) with high probability. 

It is apparent from this discussion that such an A should be hard to simulate numerically, 
for otherwise / would be computable via a numerical simulation of A. On the other hand, 
"hardness to simulate" does not immediately imply "computational hardness". Consider 
for example a fair coin. It is virtually impossible to simulate a coin toss numerically due 
to the extreme sensitivity of the process to small changes in initial conditions. Despite its 
unpredictability, a fair coin toss cannot be used to compute "hard" functions, because it is 
not robust. In fact, due to noise, for any initial conditions which puts the coin far enough 
from the ground, we know the probability distribution of the outcome: 50% "heads" and 
50% "tails". Another example where "theoretical hardness" of the wave equation does not 
immediately imply a violation of the CT is presented in [WZ02J. 

This leads to a question that is essentially equivalent to the CT: 

(*) Is there a robust physical system that is hard to simulate numerically? 

This is a question that can be formulated in the framework of bit-computability. Since 
the only numerical simulations a computer can perform are bit simulations, hardness of some 
robust system A for the bit model implies a positive answer for (*). On the other hand, 
proving that all computationally hard systems are inherently highly unstable would yield a 
negative answer to this question. 

Note that even if (*) has a positive answer and CT does not hold, and there exists 
some physical device A that can compute an uncomputable function /, it does not imply 
that this device could serve some "useful" purpose. That is, it might be able to compute 
some meaningless function /, but not any of the "interesting" undecidable problems, such 
as solving Diophantine equations or the halting problem. 
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